{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ur8xi4C7S06n"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAPoU8Sm5E6e"
      },
      "source": [
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/ai-platform-samples/blob/master/ai-platform/notebooks/unofficial/AI_Platform_Custom_Container_Prediction_sklearn.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvgnzT1CKxrO"
      },
      "source": [
        "## Overview\n",
        "\n",
        "This tutorial walks through building a custom container to serve a scikit-learn model on AI Platform Predictions. You will use the FastAPI Python web server framework to create a prediction and health endpoint.\n",
        "You will also cover incorporating a pre-processor from training into your online serving.\n",
        "\n",
        "\n",
        "### Dataset\n",
        "\n",
        "This tutorial uses R.A. Fisher's Iris dataset, a small dataset that is popular for trying out machine learning techniques. Each instance has four numerical features, which are different measurements of a flower, and a target label that\n",
        "marks it as one of three types of iris: Iris setosa, Iris versicolour, or Iris virginica.\n",
        "\n",
        "This tutorial uses [the copy of the Iris dataset included in the\n",
        "scikit-learn library](https://scikit-learn.org/stable/datasets/index.html#iris-dataset).\n",
        "\n",
        "### Objective\n",
        "\n",
        "The goal is to:\n",
        "- Train a model that uses a flower's measurements as input to predict what type of iris it is.\n",
        "- Save the model and its serialized pre-processor\n",
        "- Build a FastAPI server to handle predictions and health checks\n",
        "- Build a custom container with model artifacts\n",
        "- Upload and deploy custom container to AI Platform Prediction\n",
        "\n",
        "This tutorial focuses more on deploying this model with AI Platform than on\n",
        "the design of the model itself.\n",
        "\n",
        "### Costs \n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* AI Platform \n",
        "\n",
        "Learn about [AI Platform (Classic)\n",
        "pricing](https://cloud.google.com/ai-platform/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ze4-nDLfK4pw"
      },
      "source": [
        "### Set up your local development environment\n",
        "\n",
        "**If you are using Colab or AI Platform Notebooks**, your environment already meets\n",
        "all the requirements to run this notebook. You can skip this step."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gCuSR8GkAgzl"
      },
      "source": [
        "**Otherwise**, make sure your environment meets this notebook's requirements.\n",
        "You need the following:\n",
        "\n",
        "* Docker\n",
        "* Git\n",
        "* Google Cloud SDK (gcloud)\n",
        "* Python 3\n",
        "* virtualenv\n",
        "* Jupyter notebook running in a virtual environment with Python 3\n",
        "\n",
        "The Google Cloud guide to [Setting up a Python development\n",
        "environment](https://cloud.google.com/python/setup) and the [Jupyter\n",
        "installation guide](https://jupyter.org/install) provide detailed instructions\n",
        "for meeting these requirements. The following steps provide a condensed set of\n",
        "instructions:\n",
        "\n",
        "1. [Install and initialize the Cloud SDK.](https://cloud.google.com/sdk/docs/)\n",
        "\n",
        "1. [Install Python 3.](https://cloud.google.com/python/setup#installing_python)\n",
        "\n",
        "1. [Install\n",
        "   virtualenv](https://cloud.google.com/python/setup#installing_and_using_virtualenv)\n",
        "   and create a virtual environment that uses Python 3. Activate the virtual environment.\n",
        "\n",
        "1. To install Jupyter, run `pip install jupyter` on the\n",
        "command-line in a terminal shell.\n",
        "\n",
        "1. To launch Jupyter, run `jupyter notebook` on the command-line in a terminal shell.\n",
        "\n",
        "1. Open this notebook in the Jupyter Notebook Dashboard."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i7EUnXsZhAGF"
      },
      "source": [
        "### Install additional packages\n",
        "\n",
        "Install additional package dependencies not installed in your notebook environment, such as NumPy, Scikit-learn, FastAPI, Uvicorn, and joblib. Use the latest major GA version of each package."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "747f59abb3a5"
      },
      "outputs": [],
      "source": [
        "%%writefile requirements.txt\n",
        "joblib~=1.0\n",
        "numpy~=1.20\n",
        "scikit-learn~=0.24\n",
        "google-cloud-storage>=1.26.0,<2.0.0dev"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wyy5Lbnzg5fi"
      },
      "outputs": [],
      "source": [
        "# Required in Docker serving container\n",
        "%pip install -U -r requirements.txt\n",
        "\n",
        "# For local FastAPI development and running\n",
        "%pip install -U \"uvicorn[standard]>=0.12.0,<0.14.0\" fastapi~=0.63\n",
        "\n",
        "# AI Platform (Classic) client library\n",
        "%pip install -U google-api-python-client"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hhq5zEbGg0XX"
      },
      "source": [
        "### Restart the kernel\n",
        "\n",
        "After you install the additional packages, you need to restart the notebook kernel so it can find the packages."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EzrelQZ22IZj"
      },
      "outputs": [],
      "source": [
        "# Automatically restart kernel after installs\n",
        "import os\n",
        "\n",
        "if not os.getenv(\"IS_TESTING\"):\n",
        "    # Automatically restart kernel after installs\n",
        "    import IPython\n",
        "\n",
        "    app = IPython.Application.instance()\n",
        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lWEdiXsJg0XY"
      },
      "source": [
        "## Before you begin"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BF1j6f9HApxa"
      },
      "source": [
        "### Set up your Google Cloud project\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
        "1. [Select or create a Google Cloud project](https://console.cloud.google.com/cloud-resource-manager). When you first create an account, you get a $300 free credit towards your compute/storage costs.\n",
        "\n",
        "1. [Make sure that billing is enabled for your project](https://cloud.google.com/billing/docs/how-to/modify-project).\n",
        "\n",
        "1. [Enable the AI Platform (Classic) API and Compute Engine API](https://console.cloud.google.com/flows/enableapi?apiid=ml.googleapis.com,compute_component).\n",
        "\n",
        "1. If you are running this notebook locally, you will need to install the [Cloud SDK](https://cloud.google.com/sdk).\n",
        "\n",
        "1. Enter your project ID in the cell below. Then run the cell to make sure the\n",
        "Cloud SDK uses the right project for all the commands in this notebook.\n",
        "\n",
        "**Note**: Jupyter runs lines prefixed with `!` or `%` as shell commands, and it interpolates Python variables with `$` or `{}` into these commands."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WReHDGG5g0XY"
      },
      "source": [
        "#### Set your project ID\n",
        "\n",
        "**If you don't know your project ID**, you may be able to get your project ID using `gcloud`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oM1iC_MfAts1"
      },
      "outputs": [],
      "source": [
        "# Get your Google Cloud project ID from gcloud\n",
        "shell_output=!gcloud config list --format 'value(core.project)' 2>/dev/null\n",
        "\n",
        "try:\n",
        "    PROJECT_ID = shell_output[0]\n",
        "except IndexError:\n",
        "    PROJECT_ID = None\n",
        "\n",
        "print(\"Project ID:\", PROJECT_ID)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qJYoRfYng0XZ"
      },
      "source": [
        "Otherwise, set your project ID here."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "riG_qUokg0XZ"
      },
      "outputs": [],
      "source": [
        "if PROJECT_ID == \"\" or PROJECT_ID is None:\n",
        "    PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dr--iN2kAylZ"
      },
      "source": [
        "### Authenticate your Google Cloud account\n",
        "\n",
        "**If you are using AI Platform Notebooks**, your environment is already\n",
        "authenticated. Skip this step."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sBCra4QMA2wR"
      },
      "source": [
        "**If you are using Colab**, run the cell below and follow the instructions\n",
        "when prompted to authenticate your account via oAuth.\n",
        "\n",
        "**Otherwise**, follow these steps:\n",
        "\n",
        "1. In the Cloud Console, go to the [**Create service account key**\n",
        "   page](https://console.cloud.google.com/apis/credentials/serviceaccountkey).\n",
        "\n",
        "2. Click **Create service account**.\n",
        "\n",
        "3. In the **Service account name** field, enter a name, and\n",
        "   click **Create**.\n",
        "\n",
        "4. In the **Grant this service account access to project** section, click the **Role** drop-down list. Type \"AI Platform\"\n",
        "into the filter box, and select\n",
        "   **AI Platform Administrator**. Type \"Storage Object Admin\" into the filter box, and select **Storage Object Admin**.\n",
        "\n",
        "5. Click *Create*. A JSON file that contains your key downloads to your\n",
        "local environment.\n",
        "\n",
        "6. Enter the path to your service account key as the\n",
        "`GOOGLE_APPLICATION_CREDENTIALS` variable in the cell below and run the cell."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PyQmSRbKA8r-"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import sys\n",
        "\n",
        "# If you are running this notebook in Colab, run this cell and follow the\n",
        "# instructions to authenticate your GCP account. This provides access to your\n",
        "# Cloud Storage bucket and lets you submit training jobs and prediction\n",
        "# requests.\n",
        "\n",
        "# If on AI Platform, then don't execute this code\n",
        "if not os.path.exists(\"/opt/deeplearning/metadata/env_version\"):\n",
        "    if \"google.colab\" in sys.modules:\n",
        "        from google.colab import auth as google_auth\n",
        "\n",
        "        google_auth.authenticate_user()\n",
        "\n",
        "    # If you are running this notebook locally, replace the string below with the\n",
        "    # path to your service account key and run this cell to authenticate your GCP\n",
        "    # account.\n",
        "    elif not os.getenv(\"IS_TESTING\") and not os.getenv(\n",
        "        \"GOOGLE_APPLICATION_CREDENTIALS\"\n",
        "    ):\n",
        "        %env GOOGLE_APPLICATION_CREDENTIALS ''"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XoEqT2Y4DJmf"
      },
      "source": [
        "### Configure project and resource names"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MzGDU7TWdts_"
      },
      "outputs": [],
      "source": [
        "REGION = \"us-central1\"  # @param {type:\"string\"}\n",
        "MODEL_ARTIFACT_DIR = \"custom-container-prediction-model\"  # @param {type:\"string\"}\n",
        "REPOSITORY = \"custom-container-prediction-sklearn\"  # @param {type:\"string\"}\n",
        "IMAGE = \"sklearn-fastapi-server\"  # @param {type:\"string\"}\n",
        "MODEL_NAME = \"sklearn_custom_container\"  # @param {type:\"string\"}\n",
        "VERSION_NAME = \"v1\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "028b4d2f6d52"
      },
      "source": [
        "`REGION` - Used for operations\n",
        "throughout the rest of this notebook. Make sure to [choose a region where Cloud\n",
        "AI Platform services are\n",
        "available](https://cloud.google.com/ai-platform-unified/docs/general/locations#feature-availability). You may\n",
        "not use a Multi-Regional Storage bucket for training with AI Platform.\n",
        "\n",
        "`MODEL_ARTIFACT_DIR` - Folder directory path to your model artifacts within a Cloud Storage bucket, for example: \"my-models/fraud-detection/trial-4\"\n",
        "\n",
        "`REPOSITORY` - Name of the Artifact Repository to create or use.\n",
        "\n",
        "`IMAGE` - Name of the container image that will be pushed.\n",
        "\n",
        "`MODEL_NAME` - Name of AI Platform Model.\n",
        "\n",
        "`VERSION_NAME` - Name of AI Platform Model version."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "62f861b68b50"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "**The following steps are required, regardless of your notebook environment.**\n",
        "\n",
        "To update your model artifacts without re-building the container, you must upload your model\n",
        "artifacts and any custom code to Cloud Storage.\n",
        "\n",
        "Set the name of your Cloud Storage bucket below. It must be unique across all\n",
        "Cloud Storage buckets. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6749c55b3f6f"
      },
      "outputs": [],
      "source": [
        "BUCKET_NAME = \"gs://[your-bucket-name]\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "58cb4f5895f0"
      },
      "source": [
        "**Only if your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2d2208676cee"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $REGION $BUCKET_NAME"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c664a5abc11a"
      },
      "source": [
        "Finally, validate access to your Cloud Storage bucket by examining its contents:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2c1b1c29f5f6"
      },
      "outputs": [],
      "source": [
        "! gsutil ls -al $BUCKET_NAME"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3c2d091d9e73"
      },
      "source": [
        "## Write your pre-processor\n",
        "Scaling training data so each numerical feature column has a mean of 0 and a standard deviation of 1 [can improve your model](https://developers.google.com/machine-learning/crash-course/representation/cleaning-data).\n",
        "\n",
        "Create `preprocess.py`, which contains a class to do this scaling:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6e74556ea0b4"
      },
      "outputs": [],
      "source": [
        "%mkdir app"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "58d843d21fa8"
      },
      "outputs": [],
      "source": [
        "%%writefile app/preprocess.py\n",
        "import numpy as np\n",
        "\n",
        "class MySimpleScaler(object):\n",
        "    def __init__(self):\n",
        "        self._means = None\n",
        "        self._stds = None\n",
        "\n",
        "    def preprocess(self, data):\n",
        "        if self._means is None:  # during training only\n",
        "            self._means = np.mean(data, axis=0)\n",
        "\n",
        "        if self._stds is None:  # during training only\n",
        "            self._stds = np.std(data, axis=0)\n",
        "            if not self._stds.all():\n",
        "                raise ValueError(\"At least one column has standard deviation of 0.\")\n",
        "\n",
        "        return (data - self._means) / self._stds\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4b816cd52f4b"
      },
      "source": [
        "## Train and store model with pre-processor\n",
        "Next, use `preprocess.MySimpleScaler` to preprocess the iris data, then train a model using scikit-learn.\n",
        "\n",
        "At the end, export your trained model as a joblib (`.joblib`) file and export your `MySimpleScaler` instance as a pickle (`.pkl`) file:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "937f21278d5d"
      },
      "outputs": [],
      "source": [
        "%cd app/\n",
        "\n",
        "import pickle\n",
        "\n",
        "import joblib\n",
        "from preprocess import MySimpleScaler\n",
        "from sklearn.datasets import load_iris\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "\n",
        "iris = load_iris()\n",
        "scaler = MySimpleScaler()\n",
        "\n",
        "X = scaler.preprocess(iris.data)\n",
        "y = iris.target\n",
        "\n",
        "model = RandomForestClassifier()\n",
        "model.fit(X, y)\n",
        "\n",
        "joblib.dump(model, \"model.joblib\")\n",
        "with open(\"preprocessor.pkl\", \"wb\") as f:\n",
        "    pickle.dump(scaler, f)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3849066a33bd"
      },
      "source": [
        "### Upload model artifacts and custom code to Cloud Storage\n",
        "\n",
        "Before you can deploy your model for serving, AI Platform needs access to the following files in Cloud Storage:\n",
        "\n",
        "* `model.joblib` (model artifact)\n",
        "* `preprocessor.pkl` (model artifact)\n",
        "\n",
        "Run the following commands to upload your files:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ca67ee52d4d9"
      },
      "outputs": [],
      "source": [
        "!gsutil cp model.joblib preprocessor.pkl {BUCKET_NAME}/{MODEL_ARTIFACT_DIR}/\n",
        "%cd .."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "480a1d88ecdb"
      },
      "source": [
        "## Build a FastAPI server"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "94af0ba5eadd"
      },
      "outputs": [],
      "source": [
        "%%writefile app/main.py\n",
        "from fastapi import FastAPI, Request\n",
        "\n",
        "import joblib\n",
        "import json\n",
        "import numpy as np\n",
        "import pickle\n",
        "import os\n",
        "\n",
        "from google.cloud import storage\n",
        "from preprocess import MySimpleScaler\n",
        "from sklearn.datasets import load_iris\n",
        "\n",
        "\n",
        "app = FastAPI()\n",
        "gcs_client = storage.Client()\n",
        "\n",
        "with open(\"preprocessor.pkl\", 'wb') as preprocessor_f, open(\"model.joblib\", 'wb') as model_f:\n",
        "    gcs_client.download_blob_to_file(\n",
        "        f\"{os.environ['AIP_STORAGE_URI']}/preprocessor.pkl\", preprocessor_f\n",
        "    )\n",
        "    gcs_client.download_blob_to_file(\n",
        "        f\"{os.environ['AIP_STORAGE_URI']}/model.joblib\", model_f\n",
        "    )\n",
        "\n",
        "with open(\"preprocessor.pkl\", \"rb\") as f:\n",
        "    preprocessor = pickle.load(f)\n",
        "\n",
        "_class_names = load_iris().target_names\n",
        "_model = joblib.load(\"model.joblib\")\n",
        "_preprocessor = preprocessor\n",
        "\n",
        "\n",
        "@app.get(os.environ['AIP_HEALTH_ROUTE'], status_code=200)\n",
        "def health():\n",
        "    return {}\n",
        "\n",
        "\n",
        "@app.post(os.environ['AIP_PREDICT_ROUTE'])\n",
        "async def predict(request: Request):\n",
        "    body = await request.json()\n",
        "\n",
        "    instances = body[\"instances\"]\n",
        "    inputs = np.asarray(instances)\n",
        "    preprocessed_inputs = _preprocessor.preprocess(inputs)\n",
        "    outputs = _model.predict(preprocessed_inputs)\n",
        "\n",
        "    return {\"predictions\": [_class_names[class_num] for class_num in outputs]}\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "150cf819687b"
      },
      "source": [
        "### Add pre-start script\n",
        "FastAPI will execute this script before starting up the server. The `PORT` environment variable is set to equal `AIP_HTTP_PORT` in order to run FastAPI on same the port expected by AI Platform Prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "69f438aca35b"
      },
      "outputs": [],
      "source": [
        "%%writefile app/prestart.sh\n",
        "#!/bin/bash\n",
        "export PORT=$AIP_HTTP_PORT"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8b62ddf1def3"
      },
      "source": [
        "### Store test instances to use later\n",
        "To learn more about formatting input instances in JSON, [read the documentation.](https://cloud.google.com/ai-platform-unified/docs/predictions/online-predictions-custom-models#request-body-details)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "b6605e9e6186"
      },
      "outputs": [],
      "source": [
        "%%writefile instances.json\n",
        "{\n",
        "    \"instances\": [\n",
        "        [6.7, 3.1, 4.7, 1.5],\n",
        "        [4.6, 3.1, 1.5, 0.2]\n",
        "    ]\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "51e149fdec1b"
      },
      "source": [
        "## Build and push container to Artifact Registry"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3bdb9a7768a5"
      },
      "source": [
        "### Build your container\n",
        "Optionally copy in your credentials to run the container locally."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fbb77f4f56c7"
      },
      "outputs": [],
      "source": [
        "# NOTE: Copy in credentials to run locally, this step can be skipped for deployment\n",
        "%cp $GOOGLE_APPLICATION_CREDENTIALS app/credentials.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "240578ec9efe"
      },
      "source": [
        "Write the Dockerfile, using `tiangolo/uvicorn-gunicorn-fastapi` as a base image. This will automatically run FastAPI for you using Gunicorn and Uvicorn. Visit [the FastAPI docs to read more about deploying FastAPI with Docker](https://fastapi.tiangolo.com/deployment/docker/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3d3a6b9ed22b"
      },
      "outputs": [],
      "source": [
        "%%writefile Dockerfile\n",
        "\n",
        "FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7\n",
        "\n",
        "COPY ./app /app\n",
        "COPY requirements.txt requirements.txt\n",
        "\n",
        "RUN pip install -r requirements.txt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "04c988201499"
      },
      "source": [
        "Build the image and tag the Artifact Registry path that you will push to."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f1e7d639b9cc"
      },
      "outputs": [],
      "source": [
        "!docker build \\\n",
        "    --tag={REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE} \\\n",
        "    ."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "147a555f6c93"
      },
      "source": [
        "### Run and test the container locally (optional)\n",
        "\n",
        "Run the container locally in detached mode and provide the environment variables that the container requires. These env vars will be provided to the container by AI Platform Prediction once deployed. Test the `/health` and `/predict` routes, then stop the running image."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "62ed2d334d0f"
      },
      "outputs": [],
      "source": [
        "!docker rm local-iris\n",
        "!docker run -d -p 80:8080 \\\n",
        "    --name=local-iris \\\n",
        "    -e AIP_HTTP_PORT=8080 \\\n",
        "    -e AIP_HEALTH_ROUTE=/health \\\n",
        "    -e AIP_PREDICT_ROUTE=/predict \\\n",
        "    -e AIP_STORAGE_URI={BUCKET_NAME}/{MODEL_ARTIFACT_DIR} \\\n",
        "    -e GOOGLE_APPLICATION_CREDENTIALS=credentials.json \\\n",
        "    {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ce629eea32fd"
      },
      "outputs": [],
      "source": [
        "!curl localhost/health"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "56986f93438e"
      },
      "outputs": [],
      "source": [
        "!curl -X POST \\\n",
        "  -d @instances.json \\\n",
        "  -H \"Content-Type: application/json; charset=utf-8\" \\\n",
        "  localhost/predict"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a29fcbbe0188"
      },
      "outputs": [],
      "source": [
        "!docker stop local-iris"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "212b2935ea12"
      },
      "source": [
        "### Push the container to artifact registry\n",
        "\n",
        "Configure Docker to access Artifact Registry. Then push your container image to your Artifact Registry repository."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8b2c683d1e2d"
      },
      "outputs": [],
      "source": [
        "!gcloud beta artifacts repositories create {REPOSITORY} \\\n",
        " --repository-format=docker \\\n",
        " --location=$REGION"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "293437024749"
      },
      "outputs": [],
      "source": [
        "!gcloud auth configure-docker {REGION}-docker.pkg.dev"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1dd7448f4703"
      },
      "outputs": [],
      "source": [
        "!docker push {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2115d0e626fc"
      },
      "source": [
        "## Deploy to AI Platform (Classic)\n",
        "\n",
        "Use gcloud CLI to create your model and model version."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4234eb122018"
      },
      "source": [
        "### Create the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "40dd8d600f37"
      },
      "outputs": [],
      "source": [
        "!gcloud beta ai-platform models create $MODEL_NAME \\\n",
        "    --region=$REGION \\\n",
        "    --enable-logging \\\n",
        "    --enable-console-logging"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "af24e58f651a"
      },
      "source": [
        "### Create the model version\n",
        "After this step completes, the model is deployed and ready for online prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d378cdf9781f"
      },
      "outputs": [],
      "source": [
        "!echo \"deploymentUri: {BUCKET_NAME}/{MODEL_ARTIFACT_DIR}\" > config.yaml"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "de550bc7e8cd"
      },
      "outputs": [],
      "source": [
        "!gcloud beta ai-platform versions create $VERSION_NAME \\\n",
        "    --region=$REGION \\\n",
        "    --model=$MODEL_NAME \\\n",
        "    --machine-type=n1-standard-4 \\\n",
        "    --config=config.yaml \\\n",
        "    --image={REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bb1210135712"
      },
      "source": [
        "## Send predictions\n",
        "\n",
        "### Using REST"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cea0d0a099a9"
      },
      "outputs": [],
      "source": [
        "!curl -X POST \\\n",
        "    -H \"Authorization: Bearer $(gcloud auth print-access-token)\" \\\n",
        "    -H \"Content-Type: application/json; charset=utf-8\" \\\n",
        "    -d @instances.json \\\n",
        "    https://{REGION}-ml.googleapis.com/v1/projects/{PROJECT_ID}/models/{MODEL_NAME}/versions/{VERSION_NAME}:predict"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2ee659ab71d1"
      },
      "source": [
        "### Using Python SDK"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f3d3a69a4975"
      },
      "outputs": [],
      "source": [
        "from google.api_core.client_options import ClientOptions\n",
        "from googleapiclient import discovery\n",
        "\n",
        "client_options = ClientOptions(api_endpoint=f\"https://{REGION}-ml.googleapis.com\")\n",
        "\n",
        "service = discovery.build(\"ml\", \"v1\", client_options=client_options)\n",
        "\n",
        "response = (\n",
        "    service.projects()\n",
        "    .predict(\n",
        "        name=f\"projects/{PROJECT_ID}/models/{MODEL_NAME}/versions/{VERSION_NAME}\",\n",
        "        body={\"instances\": [[6.7, 3.1, 4.7, 1.5], [4.6, 3.1, 1.5, 0.2]]},\n",
        "    )\n",
        "    .execute()\n",
        ")\n",
        "\n",
        "if \"error\" in response:\n",
        "    raise RuntimeError(response[\"error\"])\n",
        "else:\n",
        "    print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "082e2b01beb8"
      },
      "source": [
        "### Using gcloud CLI"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "368fe2a1b6a7"
      },
      "outputs": [],
      "source": [
        "!gcloud beta ai-platform predict \\\n",
        "  --region=$REGION \\\n",
        "  --model=$MODEL_NAME \\\n",
        "  --json-request=instances.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TpV-iwP9qw9c"
      },
      "source": [
        "## Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sx_vKniMq9ZX"
      },
      "outputs": [],
      "source": [
        "# Delete the model version\n",
        "!gcloud ai-platform versions delete $VERSION_NAME \\\n",
        "    --region=$REGION \\\n",
        "    --model=$MODEL_NAME \\\n",
        "    --quiet\n",
        "\n",
        "# Delete the model\n",
        "!gcloud ai-platform models delete $MODEL_NAME \\\n",
        "    --region=$REGION \\\n",
        "    --quiet\n",
        "\n",
        "# Delete the container image from Artifact Registry\n",
        "!gcloud artifacts docker images delete \\\n",
        "    --quiet \\\n",
        "    --delete-tags \\\n",
        "    {REGION}-docker.pkg.dev/{PROJECT_ID}/{REPOSITORY}/{IMAGE}"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "AI_Platform_Custom_Container_Prediction_sklearn.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
